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Abstract 

Optimal  control  and  estimation  problems  arc  currently  solved  by  embedding  a  differential  equation  solver 
into  the  optimization  strategy.  The  optimization  algorithm  chooses  the  control  profile,  or  parameter 
estimates,  and  requires  the  differential  equation  routine  to  solve  the  equations  and  evaluate  the  objective  and 
constraint  functionals  at  each  step.  Two  popular  methods  for  optimal  control  that  follow  this  strategy  are 
Control  Vector  Iteration  (CVI)  and  Control  Vector  Parameterization  (CVP).  CV1  requires  solution  of  the 
Eulcr-Lagrangc  equations  and  minimization  of  the  Hamiltonian  while  CVP  involves  repeated  differential 
equation  solutions  driven  by  direct  search  optimization'll]) 


Both  methods  can  be  prohibitively  expensive  even  for  small  problems J^ccause  they  tend  to  converge  slowly 
and  require  solution  of  differential  equations  at  each  iteration.  -We  introduce^  method  that  avoids  this 
requirement  |>y  simultaneously  converging  to  the  optimum  while  solving  the  differential  equations.  To  do 
this,  faapffy  orthogonal  collocation  to  the  system  of  differential  equations  and  convert  them  into  algebraic 
ones.  ”  WethcTLaj$y  an  optimization  strategy  that  docs  not  require  satisfaction  of  equality  constraints  at  each 

Ke  \S 

iteration.  Here  the  method  is  applied  to  a  small  initial  value  optimal  control  problem,  although  -we-are  by  no 
means  restricted  to  problems  of  this  type. 


l".  7.  .' 
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1 .  Method  Development 

Unlike  finite  difference  ODE  solvers,  orthogonal  collocation  applies  a  polynomial  approximation  to  the 
differential  equation  and  requires  satisfaction  of  the  equation  at  discrete  collocation  points,  the  zeros  of 
orthogonal  polynomials  [2].  The  polynomial  solution  is  thus  a  continuous  function  of  t  that  is  often  as 
accurate  as  a  finite  difference  solution  using  many  more  points.  For  example,  the  polynomial  approximation 

for  initial  value  problems  defined  over  a  finite  interval  is: 

n 

yn  -  y0+-t  £  atpi-W  0) 

/= i 

where 

a{  *  unknown  coefficients 
PH  -0-U  order  I  .egendre  polynomial. 

The  coefficients  a;  in  (l)can  be  found  by  substituting^/)  into  the  initial  value  problem:  $  =  JXy,t)\  ,y(0)= 
y0  and  solving:  — f(yvi)=Q.  at  discrete  points  tt  which  arc  the  roots  of  /*„(/)= 0.  TTiis  system  can  be 

solved  by  Gaussian  elimination  if f(i,y)  is  linear  or  by  Newton's  method  if  Jlt.y)  is  nonlinear.  In  cither  case, 
the  system  of  ODE'S  is  converted  into  algebraic  equations. 

Recently,  optimization  techniques  have  been  developed  [3.  4]  that  solve  algebraic  equality  constrained 
problems  without  requiring  satisfaction  of  the  equations  at  each  iteration.  Among  the  most  promising  of 
these  is  the  Successive  Quadratic  Programming  (SQP)  (4]  algorithm.  Loosely  speaking,  this  method  linearizes 
inequality  and  equality  constraints  and  constructs  a  convex  quadratic  objective  function  from  gradients  of  the 
objective  and  constraint  functions.  The  resulting  quadratic  program  (QP)  can  be  solved  using  any  standard, 
ftnite-sicp  QP  algorithm  [S,  6],  Solution  of  the  QP  determines  the  search  direction  while  a  one-dimensional 
minimization  along  this  direction  locates  the  next  point  Here,  only  the  linearized  sets  of  equality  constraints 
are  solved  by  the  QP.  As  SQP  converges  to  die  optimum,  the  solution  of  the  linearized  sets  converges  to  the 
solution  of  the  equality  constraints.  If  no  degrees  of  freedom  arc  present  for  optimization,  the  SQP  algorithm 
reduces  to  Newton’s  method. 

Because  we  no  longer  need  to  solve  the  collocation  equations  at  each  iteration,  this  Simultaneous 
Optimization  and  COLl.ocation  (SOCOLI.)  method  can  prove  to  be  very  powerful  for  optimization  problems 
described  by  differential  equations. 


Consider  the  following  initial  value  optimization  (Mayer  optimal  control)  problem: 


(2) 


Min  F\y(iA.  u(lj),  q.  I A 
{**«} 

Si  %-  =Ay^,q.i) 
h(y,u,q,t)  =  0 


g(y.u,q,t)  S  0 

where 

u(t)  *  continuous  control  variables 
y(t)  -  state  variables 
q  -constant control  parameters 
h  -  algebraic  equality  constraints 

g  *  algebraic  inequality  constraints 

F  -  objective  functional 

tf  *  fixed  final  time 

N 

We  can  substitute  polynomial  approximations  yM  a,PH^  for  y(t)  and  include  the  coefficients  at 

as  decision  variables  in  the  optimization  problem.  However  it  is  difficult  to  provide  bounds  and  starting 
points  for  these  coefficients  because  they  have  no  physical  significance,  thus  no  apriori  estimated  ranges.  To 
remedy  this,  an  equivalent  formulation  is  found  by  writing  the  approximation  as  a  Lagrange  interpolation 
polynomial: 

a  a 

^0*  gM< 0  where  /,</)=  n</- (,)/«,- f,).  (3) 

Here  t0  *  0  and  tp  i  *  1,  n  are  zeros  of  an  nth  order  Legendre  polynomial  defined  from  0  to  tf.  Choosing  yf 
my  Jit,)  ss  decision  variables  for  the  optimization  problem,  it  is  now  much  easier  to  supply  meaningful 
bounds  and  starting  points  from  physical  insight  about  the  problem.  Other  decision  variables  arc  the  constant 
parameters,  q  (if  present)  and  coefficients  u,  of  the  polynomial  approximation  to  the  control  profiles.  The 

control  profiles  may  be  approximated  by: 

a  a 

«»*  where  Tt(i)=  Jj {t-tj)/{trij) 

aMbough  we  are  not  limited  to  this  form. 


(4) 
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This  formulation  easily  accomodates  algebraic  inequality  and  equality  constraints,  g  and  h,  which  are  often 
difficult  to  handle  with  control  vector  iteration  [7]. 

Having  defined  the  set  of  decision  variables  x  =  Uj.  qj,  we  write  the  ODH’s  as  algebraic  equalities  at  n 
collocation  points.  If  additional  constraints  g,  h  at  other  points  in  time  tp  are  present,  these  arc  included  in  the 
nonlinear  program  also.  By  substituting  equations  (3)  and  (4)  into  (2),  the  approximated  problem  now 
becomes: 

Min  i\y „U/),  u„(if),tfq) 

{yfur4\ 

S.L  r,  =  dy^/dl  -  J\yb  ub  tj.q)=Q  /=  1,  n 
h(tp,y^tp),  u£tp),  q)  —  0 

gUp.y^lp).  u^lplq)  5  0 

ytiyti,  yu 

u,SutiuM  (5) 

or  equivalently: 

Min  F(x) 

W 

S.L  r(jr)  =  0 
t(x)S  0 

Xj  SxSXf, 

(6) 

We  now  simply  apply  the  SQP  method  to  (6).  At  each  iteration,  k,  SQP  sets  up  and  solves  the  QP: 

Min  VF(xhTd+\<f&d 
14 

SX  r(jc*)  +  V  r{x*)Td  =  0 

'  A(Jt*)+ VA(jr*)*d_  0 


*(**)+  Vg(x*fdsO 


to  determine  the  search  direction,  d,  for  the  next  iterate  xk+1.  Here  the  Bk  matrix  is  constructed  from 
gradient  information  at  previous  iterations. 


This  approach  yields  an  implicit  orthogonal  collocation  solution  to  the  ODK's,  is  easy  to  apply  and 
converges  to  the  optimum  supcrlincarly.  To  illustrate  performance  of  this  method,  consider  the  following 
optimal  control  problem  [1]. 


2.  Example 

A  batch  reactor  operating  over  a  one  hour  period  produces  two  products  according  to  die  parallel  reaction 
mechanism:  A  — »  B,  A  — »  C.  Both  reactions  are  irreversible  and  first  order  in  A,  and  have  rate  constants 
given  by: 


k;  =  kj„  expi-Ej/RT}  i=1.2 


(8) 


where 

k,o  =  10‘/s 


Ic^  =  5.10]1/s 


E1  =  10000  cal/gmol 
Ej  =  20000  cal/gmol 

The  objective  is  to  find  the  temperature-time  profile  that  maximizes  the  yield  of  B  for  operating  temperatures 
below  282°F.  Therefore,  control  problem  is: 

Max  5(10) 


S.L  #  =  -(*,+  kJA 


§=k,A 
A(0)  =  A0 


m=  o 

T  &  2&2°F  (9) 

Introducing  the  following  transformations: 

* =  ■£-  Xi  =  “  *  *i  -y-  =  *2  (10) 

simplifies  the  optimization  problem  to: 
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Max  >2(1.0) 

>-i=  -(u+u1/2)yi 

>i(0)  =  1.  y/O)  =  0 

0S«S5  (11) 

Note  that  the  control  variable  u(t)  is  the  rate  constant  kL  and  directly  corresponds  to  temperature.  Itiis  insight 
eliminates  the  exponential  terms  and  simplifies  the  structure  of  the  problem. 

The  simultanious  optimization  and  collocation  (SOCOI.I.)  method  was  compared  to  the  two  traditional 
methods  for  solving  optimal  control  problems:  control  vector  iteration  (CVI)  and  control  vector 
parameterization  (CVP).  With  CVI.  the  Hamiltonian: 

H m  -X,(«  +  tf/2)yx  +  \lUyi  (12) 

is  maximized  with  rcpcct  to  u(t).  Given  an  initially  guessed  control  profile,  the  algorithm  first  integrates  the 
state  equations  forward  in  time  to  get  y,  then  the  adjoint  equations  (K  =  —  d  H/d  y)  backward  in  time  to 
obtain  A-  The  control  profile,  u(t).  is  then  updated  using  dH/du.  Here  we  apply  the  conjugate  gradient 
algorithm  of  Iasdon  cl  al.  [8],  with  the  method  of  Pagurek  and  Woodsidc  [9]  used  to  handle  control  bounds. 
The  CVP  method  was  much  more  straightforward;  the  control  profile  was  defined  by  feedback  terms  in  yr 
dial  is  u  =  bo  +  bj  yj  +  b2  y*.  Optimal  values  for  bt  were  found  by  applying  the  Complex  method  of  Box 
pO]  to  the  optimization  problem.  Both  CVI  and  CVP  used  the  DGEAR  subroutine  [11],  a  version  of  Gear’s 
method  for  stiff  initial  value  problems,  to  solve  the  ODE'S.  For  this  problem  the  converged  solution  to  CVI 
can  be  made  arbitrarily  accurate  by  specifying  tolerances  for  the  ODE  solver  and  the  optimality  conditions. 
(All  tolerances  in  this  study  were  set  to  10"*.)  With  CVP,  the  final  control  profile  is  optimal  only  with  respect 
to  a  linear  combination  of  basis  functions  and  can  never  be  better  than  with  CVI. 

Using  the  SOCOIJ.  method,  the  problem  was  first  approximated  by  1  ^grange  interpolation  polynomials 
for  n  ranging  firom  1  to  5.  Because  the  control  profile  is  only  specified  at  n  collocation  points,  its 
approximating  polynomial  (4)  is  of  one  order  less  than  the  polynomial  for  y.  To  provide  a  fair  comparison 
between  CVI,  CVP  and  SOCOLU  the  starting  points  for  y2  and  y2  were  set  to  values  of  the  initial  feasible 
simulation  at  the  collocation  points.  The  three  methods  were  compared  for  two  initially  guessed  constant 
profiles:  u(t)  =  1.0  and  u(t)  =  5.0.  These  correspond  to  operations  at  temperatures  of  196°F  and  282°F, 
respectively,  for  the  entire  reaction  time. 


yjMr 


V*'*v: 


The  results  arc  presented  in  Table  1.  Starting  from  cither  profile,  the  CV1  and  SOCOLL  methods 
converged  to  optimal  points.  The  SOCOL.L  methods  were  much  faster  and  their  maxima,  as  n  increases, 
approach  the  optimum  obtained  with  CVI  from  above.  Note  that  the  5  point  SOCOLL  solution  is  within 
0.5%  of  the  CVI  optimum,  although  CVI  required  from  2.5  to  8.7  times  as  much  computational  effort 

Surprisingly,  the  CVP  method  did  not  require  excessive  computational  effort  This  is  due  to  the  small 
number  of  decision  variables  and  the  case  in  solving  the  equations  with  DGKAR.  It  should  also  be 
mentioned  that  three  additional  runs  of  the  CVP  method  were  needed  in  order  to  establish  judicious  bounds 
for  values  of  bj.  These  arc  not  shown  in  'fable  1.  Often,  these  methods  can  be  prohibitive  because  direct 
search  methods  arc  slow  to  converge,  especially  for  large  problems,  and  the  bounds  on  ty  cannot  be  specified  a 
priori.  'ITie  CVP  optimum  is  0.8%  lower  than  the  CVI  maximum  even  though  CVP  solved  the  differential 
equations  as  CVI  did.  Moreover,  the  CVP  objective  can  never  reach  the  CVI  optimum  because  the  functional 
choice  for  u(t)  is  incomplete.  Since  the  SOCOL.L  approximation  approaches  the  true  optimum  as  n  increases, 
its  results  arc  not  as  restrictive  as  CVP's. 

Table  2  compares  values  of  the  optimal  control  profile  for  CVI,  CVP  and  5  point  SOCOLL  at  the 
collocation  points.  Here  the  agreement  between  CVI  and  SOCOLL  is  much  better  than  with  CVI  and  CVP. 
Figure  1  shows  the  optimal  control  profiles  for  the  methods  compared  above.  Here  we  observe  a  limitation  of 
SOCOLL  As  with  other  collocation  methods.  SOCOLL  cannot  approximate  steep  gradients  well  unless 
higher  order  terms  or  collocation  on  finite  elements  arc  used.  Also,  constraints  on  the  control  trajectory  can 
easily  be  applied  and  satisfied  at  collocation  points  but  may  not  be  satisfied  elsewhere  (e.g.,  between  0.95  and 
1.0).  Again,  collocation  on  finite  elements  embedded  in  SOCOLL  can  handle  this  limitation.  For  this 
example,  however,  we  can  obtain  a  better  solution  through  some  insight  into  the  control  trajectory.  We  note 
that  the  value  of  ut  is  5.0  at  the  last  collocation  point  Since  the  trajectory  defined  by  the  Lagrange 
interpolation  polynomial  violates  the  upper  bound  on  u„  between  the  last  collocation  point  and  1.0,  we  merely 
"dip"  u(t)  by  defining  it  as: 

u(i)=  min  (5.0,  u  (/)) 

n 

Since  uQ  >  5.0  only  after  the  last  collocation  point  (0.953),  the  control  profile  can  be  clipped  without 
affecting  the  collocation  constraints  or  continuity  and  differentiability  (wrt  x)  of  the  objective  function.  We 
applied  the  following  clipping  procedure: 
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if  u  (1.0)  t  5.0,  find  /  e  [0.953, 1.0)  where  u=  5. 

If  C  n 

Set  u(t)  =  5  for  1 1  [/f,  1J;  the  variables  ^(/)  and  >:(/).  1 1  [/e,  1  ]  arc  calculated  by: 
yxU)-yxU}exp{-n.S{t-t}\ 

yjil)  =  yp)  +  (-5/17.5)^  [exp{- 17.5  (/-  ^}-l]  (13) 

since  the  differential  equations  arc  linear  once  u  is  constant  'fhc  clipped  SOCOI.L  optimum  is  within  0.1% 
of  the  CVI  optimum.  Agreement  with  CV1  at  collocation  points  is  not  as  good  as  with  the  undipped 
SOCOLI.  method,  but  its  control  trajectory  is  bounded  between  0  and  5  and  agrees  reasonably  well  with  CVI 
and  Figure  1. 

These  results  arc  indicative  of  applications  to  other  initial  value  optimal  control  problems.  The  accuracy  of 
the  solution  is  limited  only  by  the  error  introduced  by  the  collocation  procedure.  Once  a  problem 
formulation  has  been  chosen  which  insures  that  collocation  can  be  applied  accurately,  then  the  accuracy  of 
the  solution  to  the  optimal  control  problem  is  subject  only  to  the  tolerance  on  the  optimality  conditions. 

The  implcmentrtion  of  the  SQP  algorithm  used  here  also  has  local  supcrlincar  and  global  convergence 
properties.  It  operates  in  a  much  smaller  space  than  the  CVI  algorithm  and  will  generally  be  more  accurate 
than  the  CVP  algorithm  because  it  is  not  as  limited  by  the  basis  functions  for  the  optimal  control  profile. 

3.  Conclusion* 

A  simple  method  has  been  described  for  efficiently  solving  dynamic  optimization  problems.  For  a  small 
optimal  control  problem,  very  good  approximate  optima  can  be  found  with  relatively  little  computational 
effort  The  formulation  presented  above  can  easily  be  extended  to  handle  collocation  on  finite  elements  (for 
stiff  systems  of  ODE’S)  as  well  as  two  point  and  other  boundary  value  problems.  A  key  point  observed  in  the 
solution  of  this  small  problem  is  that  the  system  of  differential  equations  is  never  solved  explicitly.  Instead, 
the  optimization  algorithm  converges  simultaneously  to  solve  the  set  of  ODE’S  and  find  the  optimal 
trajectory.  Thus,  the  often  considerable  computational  effort  of  solving  a  set  of  ODE’s  at  each  iteration  is 
saved. 

To  conclude,  we  note  the  following  points: 

l.Thc  SOCOI.L  strategy  handles  stiff  ordinary  differential  equations  without  difficulty  since  it 
yields  an  implicit  collocation  solution. 

1  The  solution  of  this  method  is  only  limited  by  the  accuracy  of  the  collocation  procedure. 


3.  The  optimization  procedure  solves  the  collocation  equations  only  once.  It  converges  to  the 
optimum  and  the  equation  solutions  simultaneously. 

4.  The  optimal  control  problem  is  thus  transformed  to  a  nonlinear  program.  Multiple  boundary 
conditions  and  point  constraints  that  cannot  be  handled  easily  with  CVI  and  CVP  present  no 
problem  within  this  framework. 

Therefore,  we  can  expect  the  SOCOI.I.  method  to  be  an  efficient  and  effective  tool  for  solving  a  wide 
variety  of  dynamic  optimization  problems.  The  results  given  here  can  be  generalized  to  larger,  more 
complicated  problems  by  applying  finite  element  collocation. 
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Table  I:  COMPARISON  OF  MEIHODS 


Stank*  Profile  u(t)  -  1. 


*  *  « 

MCUIOQ 

CpU  Secs.* 

Optimum 

No.  Iterations 

Ipt  SOCOLL 

0.84 

0.66667 

9 

2  pt  SOCOLL 

1.44 

0.59438 

11 

3  pt  SOCOLL 

S.64 

0.59308 

30 

4  pt  SOCOLL 

11.83 

0.57858 

41 

5  pt  SOCOIX 

17.92 

037661 

44 

5  pt  SOCOLL 

14.12 

0.57263 

30 

(clipped) 

CVI 

45.12 

0.57349 

20** 

CVP 

30.07 

0.56910 

377*** 

Starting  Profile  u(t)  =  S. 


Method 

CPU  Sees.* 

Optimum 

No.  Iterations 

lpt  SOCOLL 

U8 

0.66667 

21 

2  pc  SOCOLL 

141 

0.59438 

20 

3  pt  SOCOLL 

9.69 

039308 

52 

4  pc  SOCOLL 

14.92 

037858 

53 

5  pc  SOCOLL 

26.06 

037661 

62 

5  pt  SOCOLL 

3160 

037275 

66 

(clipped) 

CVI 

226J5 

037322 

58** 

CVP 

18.61 

036910 

213*** 

*  Execution  Times,  DEC-20  Computer,  Carncgic-Mcllon  Computation  Center 

**  Number  ofCVI  Profile  Updates 

*  Number  of  Objective  Function  Calls 


TaMc  2:  OPTIMAL  PROFILE  AT  COLLOCATION  POINTS 


t  CVI  5  pt  SOCOI.L  S  ptSOCOl.l.  CVP 

(clipped) 


0.0469  0.76702 

02308  087847 


iXm  t  i 


07692 

0.9S31 


0.76074 

0.84027 

1.16616 

1.66126 

S.00000 


0.78692  0.83969 


0.97820 

1.04957 

2J0851 

4.99738 


0.77699 
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